11063 0 true 2009-05-27T11:38:42-04:00 11063 11 export-folding-is-not-utf-8-safe 110629 30941 resolved 2009-05-28T13:32:20-04:00 8551 Fjan Rick DeNatale Rick DeNatale http://rick_denatale.lighthouseapp.com/projects/30941/tickets/11 The folding which happens when exporting a calendar currently splits at a fixed character position. This may cause a UTF-8 string with multi-byte characters to be split producing invalid UTF-8 data. The folding which happens when exporting a calendar currently splits at a fixed character position. This may cause a UTF-8 string with multi-byte characters to be split producing invalid UTF-8 data. <div><p>The folding which happens when exporting a calendar currently splits at a fixed character position.</p> <p>This may cause a UTF-8 string with multi-byte characters to be split producing invalid UTF-8 data.</p></div> 11063 0 The folding which happens when exporting a calendar currently splits at a fixed character position. This may cause a UTF-8 string with multi-byte characters to be split producing invalid UTF-8 data. <div><p>The folding which happens when exporting a calendar currently splits at a fixed character position.</p> <p>This may cause a UTF-8 string with multi-byte characters to be split producing invalid UTF-8 data.</p></div> false 2009-05-27T11:38:42-04:00 11063 --- {} 11 export-folding-is-not-utf-8-safe 0 30941 new 2009-05-27T11:38:42-04:00 11063 Rick DeNatale Rick DeNatale Rick DeNatale http://rick_denatale.lighthouseapp.com/projects/30941/tickets/11 11063 0 Very clever. You only checked the remainder and it worried me that there could be a UTF-8 character that would split into something that was still valid, but I checked the UTF-8 spec and that doesn't seem possible. In fact, you can also determine if you are dealing with a split-in-the-middle character by simply checking if the top two bits are '10', which may be a bit cheaper than String.unpack <div><p>Very clever. You only checked the remainder and it worried me that there could be a UTF-8 character that would split into something that was still valid, but I checked the UTF-8 spec and that doesn't seem possible. In fact, you can also determine if you are dealing with a split-in-the-middle character by simply checking if the top two bits are '10', which may be a bit cheaper than String.unpack</p></div> false 2009-05-28T07:55:09-04:00 11063 --- {} 11 export-folding-is-not-utf-8-safe 0 30941 new 2009-05-28T07:55:11-04:00 8551 Fjan Rick DeNatale Rick DeNatale http://rick_denatale.lighthouseapp.com/projects/30941/tickets/11 11063 0 I suspect that String.unpack is actually quicker than checking the byte with ruby, since it's a C 'primitive' In any case I doubt that it makes much performance difference, I trust String#pack to get the utf-8 rules right. <div><p>I suspect that String.unpack is actually quicker than checking the byte with ruby, since it's a C 'primitive'</p> <p>In any case I doubt that it makes much performance difference, I trust String#pack to get the utf-8 rules right.</p></div> true 2009-05-28T08:34:05-04:00 11063 --- :state: new 11 export-folding-is-not-utf-8-safe 0 30941 resolved 2009-05-28T08:34:09-04:00 11063 Rick DeNatale Rick DeNatale Rick DeNatale http://rick_denatale.lighthouseapp.com/projects/30941/tickets/11 11063 0 By the way, here is what I came up with, slightly shorter but a bit less elegant: class String def ical_wrap if (l=self.length)>75 then x=75 begin if self[x]&192!=128 then self.insert(x,"\r\n ") # change for Ruby 1.9 elsif self[x-=1]&192!=128 then self.insert(x,"\r\n ") else self.insert(x-=1,"\r\n ") end x+=77;l+=3 end while x<l end self end end <div><p>By the way, here is what I came up with, slightly shorter but a bit less elegant:</p> <p>class String<br> def ical_wrap</p> <pre> <code>if (l=self.length)&gt;75 then x=75 begin if self[x]&amp;192!=128 then self.insert(x,"\r\n ") # change for Ruby 1.9 elsif self[x-=1]&amp;192!=128 then self.insert(x,"\r\n ") else self.insert(x-=1,"\r\n ") end x+=77;l+=3 end while x&lt;l end self</code> </pre> <p>end end</p></div> true 2009-05-28T13:32:19-04:00 11063 --- {} 11 export-folding-is-not-utf-8-safe 0 30941 resolved 2009-05-28T13:32:20-04:00 8551 Fjan Rick DeNatale Rick DeNatale http://rick_denatale.lighthouseapp.com/projects/30941/tickets/11